Benefits of using the eye-tracking method for qualitative observation of students’ multiplechoice physics tasks solution process

Metoda oční kamery se v rámci různých studií ukázala jako vhodnou pro studování kognitivních procesů žáků. V článku se zaměříme na výhody použití metody sledováním žáků středních škol při řešení testu s otázkami s výběrem odpovědi. Konkrétně se zaměříme na kvalitativní analýzu dat týkajících se pohybů očí dvou žákyň řešících úlohy z fyziky. Obě žákyně řešily všechny testové úlohy nesprávně, obě navíc vybraly jako odpověď stejné alternativy. V případě klasického zadání testu (vyplňování papírových dotazníků), bychom dostali od obou žákyň naprosto stejný výstup. Data z oční kamery nám však ukázala jak podobné přístupy řešení, tak i důležité odlišnosti, na základě kterých by pak vyučující pracovali s každou dívkou odlišně. Obě dívky četly zadání pozorně a věnovaly důraz klíčovým pojmům. Je tedy zřejmé, že špatná volba odpovědi nejde na vrub nedostatečného čtení zadání. Obě dívky zřejmě mají typickou miskoncepci týkající se 1. Newtonova zákona. Liší se zejména v jejich jistotě výběru odpovědi. Eva byla v odpovídání rychlejší a zabývala se zejména možností, kterou nakonec vybrala jako odpověď. Naproti tomu Jana se velmi často vracela k zadání, často nejprve přemýšlela o odpovědi, kterou nakonec nevybrala jako správnou. U Evy jsme pozorovali netypické vertikální sakády nad vybranými klíčovými pojmy. Jana při pohledu na spíše ilustrativní obrázek projevila typickou miskoncepci “graf jako obrázek”. Detailní analýzou jsme ukázali, že souhrnná data na tzv. oblast zájmu mohou být zavádějící, pokud je tato oblast příliš velká (např. jedna alternativa v testové otázce). Dále jsme představili možné indikátory pro kvalitativní analýzu a možnosti vizualizace zpětných sakád.As previous studies have shown, the eye-tracking method is useful for monitoring cognitive processes. In this study, we focus on the benefits of analysing high school students’ problem-solving processes when completing a multiple-choice test. Specifically, we provide a qualitative analysis of eye-movements of two low prior knowledge students as they solve two physics tasks. Both students scored zero on the test and chose the same options. In a paper-pencil administration of such a test, we would receive the same results from both of them. Via analysis of the eye-tracking data, we have shown that there are several similarities and differences between their solving approaches. Particularly, they both emphasized the key terms in the stem, meaning they did not provide incorrect answers due to improper stem reading. Both students showed very typical misconceptions concerning understanding Newton’s First Law. However, they differ in their confidence. Eve arrived at her answer quicker, but Jane needed to reread the stem many times. We observed atypical saccadic eye movements when Eve read the stems of the tasks and much longer fixations when she suddenly switched to her answers. Jane showed “graph as a picture error” when interpreting an illustrative picture in the stem. Based on our analysis, we demonstrate that summarized data on areas of interest (AOIs) can be misleading when AOIs are too large (e.g., each test option). We also introduce several indicators appropriate for a qualitative analysis, and we present new visualisations for in-word and between-words saccades

EYE-TRACKING THE EFFECTS OF REPRESENTATION ON STUDENTS’ PROBLEM SOLVING APPROACHES

This study used an eye-tracking method to explore students’ approaches to solving the same task in different representations and the differences between students who answered correctly and those who answered incorrectly when solving the problems. Thirty-one upper secondary students took part in the study. According to the results of this study, the representation has a slight effect on students’ problem solving processes. Students with the incorrect answers paid more attention to the parts of the tasks with information irrelevant to determining the solution to the task

Experimenting with the structured assessment dialogue in physics and mathematics classes

Teacher–student discussions are central in assessing students’ skills and knowledge. This study deals with formal, pre-planned discussions to assess students’ inquiry competences. The method is called the structured assessment dialogue (SAD) which combines formative and summative assessment. Six SAD sessions (4 lower secondary physics and 2 upper secondary math) were implemented at the end of inquiry lessons. The SAD sessions were video recorded and the discussions were analyzed using theory based categories and networks. The results showed that the truly novel SAD method in Finnish classrooms challenged teachers and students. For example, students usually gave lower order answers although the teacher asked higher order questions. Lower and upper secondary level dialogues had different characteristics. In the lower secondary, the teacher struggled to engage students by using what students themselves had said. In the upper secondary, the teacher focused on addressing a particular idea, which turned out to be difficult for the student. Despite the challenges, SAD could be a useful method in Finnish schools if teachers and student would get used to it

English

Educational field can take advantage of the improvements of Automatic Speech Recognition (ASR), since we can apply ASR algorithms in non-ideal conditions such as real classrooms. In the context of [name deleted to maintain the integrity of the review process], conceptual networks are studied. The networks relate key concepts used by the teacher in his/her discourse. In the present study, quantitative metrics are provided, such as centrality measures and PageRank, which can be used to analyse the conceptual networks. With a case-study design, two teachers’ classes are described quantitatively and qualitatively using the metrics, suggesting that PageRank could be a good metric to find differences in teachers’ discourse. Finally, we discuss about the potential of this kind of analysis

Huomion kiinnittyminen koko luokan simulaatio-opetuksessa - tapaustutkimus katseenseurantaa käyttäen

Nykyteknologian avulla opiskelijoiden huomionkiinnittymistä opetuksen aikana voidaan tutkia esimerkiksi katseenseurantalaseja käyttäen. Tätä kautta saadaan tietoa opiskelijoiden kognitiivisista prosesseista ja opettajan toiminnan vaikutuksesta niihin. Tässä tutkimuksessa seurattiin erään lukiolaisen katsetta koko luokan simulaatio-opetuksen aikana. Analyysi keskittyi opettajan toiminnallaan antamien vihjeiden merkitykseen huomion kiinnittymiselle. Opettaja ja opiskelija myös haastateltiin. Opiskelija seurasi melko hyvin opettajan antamia vihjeitä; opettajan vuorovaikutus simulaation kanssa sekä selvät kysymykset saivat opiskelijan huomion kiinnittymään vihjeiden mukaisesti. Tulevaisuudessa tulisi kerätä lisää dataa eri konteksteissa tulosten vahvistamiseksi.Nykyteknologian avulla opiskelijoiden huomionkiinnittymistä opetuksen aikana voidaan tutkia esimerkiksi katseenseurantalaseja käyttäen. Tätä kautta saadaan tietoa opiskelijoiden kognitiivisista prosesseista ja opettajan toiminnan vaikutuksesta niihin. Tässä tutkimuksessa seurattiin erään lukiolaisen katsetta koko luokan simulaatio-opetuksen aikana. Analyysi keskittyi opettajan toiminnallaan antamien vihjeiden merkitykseen huomion kiinnittymiselle. Opettaja ja opiskelija myös haastateltiin. Opiskelija seurasi melko hyvin opettajan antamia vihjeitä; opettajan vuorovaikutus simulaation kanssa sekä selvät kysymykset saivat opiskelijan huomion kiinnittymään vihjeiden mukaisesti. Tulevaisuudessa tulisi kerätä lisää dataa eri konteksteissa tulosten vahvistamiseksi

Visual cues improve students’ understanding of divergence and curl: Evidence from eye movements during reading and problem solving

The coordination of multiple external representations is important for learning, but yet a difficult task for students, requiring instructional support. The subject in this study covers a typical relation in physics between abstract mathematical equations (definitions of divergence and curl) and a visual representation (vector field plot). To support the connection across both representations, two instructions with written explanations, equations, and visual representations (differing only in the presence of visual cues) were designed and their impact on students’ performance was tested. We captured students’ eye movements while they processed the written instruction and solved subsequent coordination tasks. The results show that students instructed with visual cues (VC students) performed better, responded with higher confidence, experienced less mental effort, and rated the instructional quality better than students instructed without cues. Advanced eye-tracking data analysis methods reveal that cognitive integration processes appear in both groups at the same point in time but they are significantly more pronounced for VC students, reflecting a greater attempt to construct a coherent mental representation during the learning process. Furthermore, visual cues increase the fixation count and total fixation duration on relevant information. During problem solving, the saccadic eye movement pattern of VC students is similar to experts in this domain. The outcomes imply that visual cues can be beneficial in coordination tasks, even for students with high domain knowledge. The study strongly confirms an important multimedia design principle in instruction, that is, that highlighting conceptually relevant information shifts attention to relevant information and thus promotes learning and problem solving. Even more, visual cues can positively influence students’ perception of course materials

Analyzing science teachers’ support of dialogic argumentation using teacher roles of questioning and communicative approaches

The purpose of this study is to investigate how teachers use different types of discourse to support dialogic argumentation. Dialogic argumentation is a collaborative process in which students construct arguments together and examine arguments presented by their peers. Science teachers can use argumentation as a vehicle to help students gain a working understanding of science content and the nature of science and its practices. Whole-class closing discussions from video-recorded lessons are analyzed to study the discourse used to support argumentation by two physics teachers in lower secondary schools. Analysis of discourse includes coding of communicative approach at the episode level and coding of teacher roles of questioning at the level of speaking turns. Student argumentation is also assessed on the basis of dialogicity and complexity of arguments. Findings characterize different ways of orchestrating argumentative discussions. Authoritative episodes were characterized by the presence of the dispenser role, with teachers retaining ownership over ideas and classroom activities to emphasize the correctness of a justification. Dialogic episodes of classroom interaction showed openness to student perspectives, but teachers’ use of questioning roles revealed different ways of orchestrating argumentative discussions. The moderator role granted ownership of ideas to students to either pursue a single student’s argument in more depth or to directly contrast opposing justifications. Less commonly used were the roles of coach and participant, which teachers used to elicit student justifications in more depth or support students in examining the arguments of their peers. Examination of discourse using multiple frameworks revealed differences in teachers’ values and the impact of the use of teacher questioning roles on student contributions to argumentative discussions

English

We propose a method that automatically describes teacher talk. The method allows us to describe and compare classroom lessons, as well as visualizing changes in teacher discourse throughout the course of a lesson. The proposed method uses a machine learning model to infer topics from school textbooks. Certain topics are related to different contents (e.g. kinematics, solar system, electricity), while others are related to different teaching functions (e.g. explanations, questions, numerical exercises). To describe teacher talk, the machine learning method measures the appearance of the inferred topics throughout each lesson. We apply the proposed method to a collection of transcripts from physics lessons, as well as discussing the potentialities of integrating the proposed method with other kinds of automatic and manual classroom lesson descriptions

The automatic analysis of classroom talk

The SMART SPEECH Project is a joint venture between three Finnish universities and a Chilean university. The aim is to develop a mobile application that can be used to record classroom talk and enable observations to be made of classroom interactions. We recorded Finnish and Chilean physics teachers’ speech using both a conventional microphone/dictator setup and a microphone/mobile application setup. The recordings were analysed via automatic speech recognition (ASR). The average word error rate achieved for the Finnish teachers’ speech was under 40%. The ASR approach also enabled us to determine the key topics discussed within the Finnish physics lessons under scrutiny. The results here were promising as the recognition accuracy rate was about 85% on average

EBook proceedings of the ESERA 2011 conference : science learning and citizenship

This ebook contains fourteen parts according to the strands of the ESERA 2011 conference. Each part is co-edited by one or two persons, most of them were strand chairs. All papers in this ebook correspond to accepted communications during the ESERA conference that were reviewed by two referees. Moreover the co-editors carried out a global reviewing of the papers.ESERA - European Science Education Research Associatio